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(N. 
^ ■ 

■ Abstract. Gaussian graphical models have become a well-recognized tool for the analy- 

■ sis of conditional independencies within a set of continuous random variables. We describe 
, , the maximal group acting on the space of covariance matrices, which leaves this model 

^ ■ invariant for any fixed graph. This group furnishes a representation of Gaussian graphical 

' models as composite transformation families and enables to analyze properties of param- 

1/^ ■ eter estimators. We use these results in the robustness analysis to compute upper bounds 

^N) ■ on finite sample breakdown points of equivariant estimators of the covariance matrix. In 

I addition we provide conditions on the sample size so that an equivariant estimator exists 

'■ with probability 1. 

I 1. Introduction 

-4— » ■ 

■ Using Gaussian graphical models is a popular way of modelling complex associations in 
^ I the multivariate continuous case. Statistical inference for this model class often depends on 

the maximum likelihood estimator of the covariance matrix. As pointed out for example 

. by Maronna, Martin, Yohai [16] the maximum likelihood estimator is typically the least 

J> I robust with respect to potential outliers in the sample space. For this reason it may be 

■ interesting to understand properties of more robust estimators for this model class. 

. Robustness issues with respect to graphical models have been rarely looked at so far, 

l/^ ■ although it has been known for some time that the classical estimators and model selection 

■ procedures are vulnerable to contaminated data [11] [13] . First approaches towards robust 
I covariance estimators for a given Gaussian model with undirected graph can be found in 

^ ■ Becker [5j and Gottard, Pacillo [12j. The authors suggested there to replace the sample 

. . . covariance matrix by the reweighted MOD estimator. Miyamura and Kano [18j propose 

_ ^ I to use an M-type estimator instead. Finegold and Drton [9j as well as Vogel and Fried 

^ ■ [23j discard the assumption of normality and consider the t-distribution and general el- 

H ! liptical distribution, respectively, to model heavy tails. Ravikumar et al. [19] as well as 



Meinshausen and Biihlmann |17j deal with the question of a robust selection of the graph 
structure by Li- regularization or penalization. 

We use symmetries of the Gaussian graphical model in order to understand the behavior 
of an arbitrary equivariant estimator of the covariance matrix. Having an explicit group 
acting on a statistical model has numerous advantages. This was first pointed out by Fisher 
in the context of the location and scale models p[0|, which then led to the definition of 
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transformation families 0], [20\ Section 6.2]. A basic idea behind the transformation family 
in statistics is that of a statistical model which is invariant under the action of a group and 
this action is assumed transitive. This representation can be then used for example in the 
study of model invariants and distributional aspects of the maximum likelihood estimator 
and other equivariant estimators. 

In the present paper we use the symmetries of Gaussian graphical models in the robust- 
ness analysis for equivariant estimators of the covariance matrix, based on a more general 
theory developed by Davies and Gather [8]. This concerns an important measure of robust- 
ness for parameter estimators, which is the high breakdown point (see for example [16j). 
Intuitively, the high breakdown point tells us what proportion of the data can be changed 
arbitrarily without arbitrarily large changes in the value of a given estimator. In this paper 
we work out explicit upper bounds on breakdown points of equivariant estimators. These 
bounds depend only on the graph Q. We introduce these results in Section [2j This also 
includes one of the three main results of this paper, which we state in Theorem 12.61 

The group of interest can be defined as follows. Let X = {Xi)i^[.^j be a random vector 
with multivariate normal distribution A/'(0, S). We assume the zero mean for simplicity. 
Denote by Sm the set of symmetric matrices in and by its subset of positive 

definite matrices. Let Q = ([m],S) be an undirected graph with set of vertices [m] := 
{!,..., m} and set of edges E C [m] x [m]. By Sg C Sm denote the linear space of 
symmetric matrices whose off-diagonal entry is zero if ^ E, and by Sg the 

subset of Sg given by all positive definite matrices. The Gaussian graphical model is the 
statistical model for X given by 

M{g) = {A/-(0,S) : g5+}. 

Hence the expression for the normal log-density is a quadratic polynomial in xi, . . . ,x 
given by 



m 



log/(x;ir) = -^log2^ + idetif-i ^ i^, 
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ij XiXj , 



where K := is the concentration matrix. 

The general linear group GLm(IR) acts on the sample space M™ by the matrix multipli- 
cation 

g • X := c/x 

for g G GL^(R). Since f{x;K) = f{gx;g~'^Kg-^), the action of GL(M'") on M"* induces 
the action on S^ given by 

g-K := g-^Kg-\ 

In this paper we are interested in the following problem. 

(1) Find all c/ G GLm(M) such that g ■ Sg = Sg. 

Since in this case also g ■ Sg = Sg , we say that g leaves M{Q) invariant (see |20^ Definition 
6.27]); the set of all such 5 is a subgroup of GLm(M) denoted by G. 
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The main concept of the paper is that of a G-equivariant estimator. Let X(„) = 
(xi, . . . ,x„) be a random sample and (M™)" the space of all random samples of size n. 
An estimator is by definition a map T„ : (M™')" — > Sg , which is defined almost every- 
where. Let 

5-X(„) := (5x1, . . . ,c/xi). 

We say that an estimator T„ is G-equivariant if T„(g • X(„)) = g ■ T„(x(„)) for every X(„) G 
(M™')"' and g ^ G whenever T„(x(„)) exists. The equivariance is an important property 
for estimators as it assures that the estimator is stable under simultaneous compatible 
transformations of the parameter and the sample space (see for example [20.j Chapter 6]). 

Our main results can be summarized as follows. For a fixed undirected graph Q = 
{[m],E) define a preorder on [m] by 

(2) i^j if N{j)U{j} C N{i)U{i}, 

where N{i) denotes the set of neighbors of i in Q. Consider the set of all matrices in 
GLm(IR) such that gij 7^ only if j ^ i. We show later in Section [3.21 that this set forms a 
subgroup of GLm(M). The preorder defined in ([2]) gives an equivalence relation on [m] by 
i ~ J if and only if i ^ j and j ^ i- Let now Q be the graph on the set of equivalence classes 
[m]/ ~ induced by Q by identifying all equivalent nodes. Hence, two equivalence classes i 
and j are connected by an edge in if i and j are neighbors in Q. We color the vertices 
of G by positive numbers representing cardinality of the corresponding equivalence class. 
The automorphisms of Q, denoted by Aut(^,c), are the automorphisms of the underlying 
graph preserving the coloring of the vertices. 

Theorem 1.1. For every undirected graph Q = ([m], E) the group G of elements satisfying 
(CP is generated by its subgroups G^ and Kvit{Q). More precisely G = G^ xi Aut(^,c), that 
is G is a semidirect product of G^ and Aut(^,c). 

A natural question is what are the conditions on n and G so that the maximum likelihood 
estimator (MLE) exists with probability one. For a discussion see the recent paper [22] 
and references therein. Another related question is, what are the conditions for n and Q 
so that there exists any G-equivariant map T„ : (M*")" — )• Sg , X(„) 1— )• Tn{xn)- Since the 
MLE is G-equivariant, these conditions are weaker than the conditions for existence of the 
MLE. 

Theorem 1.2. Let Q = {[rn\,E) be an undirected graph. There exists a G-equivariant 
estimator Tn : (M'")" — )• Sg if and only n > maxjg[„] \{j G [m] : j ^ i}\. 

The paper is organized as follows. In Section [2] we formally introduce the concept of a 
breakdown point. We also link the computation of breakdown points to the symmetry of a 
statistical model. We also state Theorem 12.61 which gives the formula for the finite sample 
breakdown point of any G-equivariant estimator of the covariance matrix. In Section [3] we 
introduce various combinatorial structures associated to Gaussian graphical models. These 
will be then used in the analysis of the connected component of the identity of the group 
G defined in ([T]). In Section [4] we complete our description of the group G, which is given 



4 



JAN DRAISMA, SONJA KUHNT, AND PIOTR ZWIERNIK 



in Theorem 11.11 Examples of various graphs and corresponding groups G are provided 
in Section [5j In Section [6] we analyze G-equivariant estimators more closely, which leads 
to the proof of Theorem 11.21 and Theorem 12.61 In Section [7] we complete our geometric 
description of the group G by a study of G-orbits of generic concentration matrices in Sg . 
In particular Theorem 17.21 gives us as a result the dimension of a generic G-orbit. We 
conclude the paper with a short discussion. 

2. Breakdown points and groups 

An important concept to measure the robustness of parameter estimators are breakdown 
points. In a simple univariate situation, if the estimator is given by the sample mean, then 
an arbitrarily large change made to one of the observations leads to an arbitrarily large 
change in the value of the estimator. If we consider the sample median, then changing one 
observation in a sample of size larger than two cannot lead to arbitrarily large changes in the 
estimator. This feature makes the median more robust to outliers in the sample. Informally 
speaking a finite sample breakdown point is the minimal number of sample points such that 
altering them effects in arbitrarily large changes in the value of an estimator. This quantity 
is usually normalized by the sample size n (see also Definitions 12.21 and 12 . 3p . So for example 
in the case of the sample mean above, the breakdown point is 1/n. In terms of robustness 
the estimator with the highest breakdown point is preferred [13j . 

We now briefly discuss the main results of [8], which link the concept of a breakdown 
point and that of symmetry groups of statistical models. This is the main motivation for 
the study of the group structure of Gaussian graphical models. To formally define the 
breakdown point in the general case we need to introduce more concepts. 

A pseudometric on the parameter space Sg is a function D : 5+ X 5+ ^ M>o such that 
for every K,L,M £S^: 

1. D{K,K) = 0, 

2. D{K,L) = D{L,K), 

3. D{K,M) < D{K,L) + D{L,M). 

The only difference between a metric and a pseudometric is that we may have D{K, L) = 
even ii K ^ L. In this paper we work with a special type of a pseudometric on Sg given 
by 

(3) D{K,,K2) = \logdet{KiK^% 

The following example illustrates the main idea of the link between the breakdown of an 
equivariant estimator to symmetry groups. 

Example 2.1. Let G be a subgroup of GLm(K) and let Gi be the set of g £ G such that 
det((7^(7) 7^ 1. For a random sample X(„') assume that g € Gi stabilizes the first n — k 
sample points but alters the remaining k points, that is gxj = Xj for i = 1, . . . ,n — k and 
fl'Xj 7^ Xj for i = k + 1, . . . ,n. Then for any Z > 1 we have g'xj = Xj if z = 1, . . . , n — fc. 
Hence (5' •X(„))i>i is a sequence of data samples which all differ from X(„) in k data points. 
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We note that if T„ is a G-equivariant estimator then for D defined in ([3]) 
D{Tn{y^(n)),Tn{g^ -y^in))) = -D(T„(x(„)),5r' • r„(x(„))) = /|logdet(g^c/)| — > oo. 

In this way we have constructed a sequence of data samples with n — k points fixed such 
that the resulting estimator can be arbitrarily far from the original estimator Tn (x(„) ) with 
respect to the pseudometric D. 

Consider now the measurable sample space (W^ ,B{W^)), where B{W^) is the Borel a- 
algebra, and the family *p of all probability measures nondegenerate with respect to the 
Lebesgue measure on this space. For any g ^ G and P € ^ we define := g ■ P hy 
P^[B) = P^g^^i^B)). This is consistent with with the action of G on M™. 

A map T : ^ — > Sg is called equivariant if T{P3) = g-T{P) for every g £ G. We assume 
that a pseudometric d is defined on ^ which satisfies 

sup d{P,Q) = 1 

and for all P,Qi,Q2 S and a, < a < 1, 

d{aP + {I - a)Qi,aP + {1 - a)Q2) < I - a. 

We also consider a pseudometric D : Sg x Sg — )■ M that satisfies sup^^^^^ D{Ki,K2) = oo. 
In this paper we constrain ourselves solely to the pseudometric in ([3]), but other distances 
can be used as well. 

Definition 2.2. The breakdown point e*(T, P, d, D) of T at the distribution P with respect 
to pseudometrics d and D is defined by 

(4) e*{T,P,d,D) = inf{e>0: sup D{T{P),T{Q)) = oo}. 

d{P,Q)<c 

We can also define a breakdown point for an estimator : (M™)" Sg. If X(„) = 
(xi, . . . ,x„) is a sample of size n, then by yn,k denote a sample obtained from X(„) by 
altering at most k of the Xj. 

Definition 2.3. The finite sample breakdown point of a G-equivariant estimator T„ at the 
sample Xj-^) is defined by 

(5) fsbp(r„,X(„),L>) = -min{A; G [n] : supZ)(r„(x(„)),r„(y„^fc)) = oo}. 
In particular fsbp(T„,X(„), Z?) > 1/n. 

Intuitively Example 12.11 shows that symmetries of a statistical model can be exploited in 
order to construct a sequence of data points such that the supremum above is infinite. Now 
we explain formally how the group G is used in ([7]) to find upper bounds for e*(T, P, d, D) 
and fsbp(T„,X(„), Z?) for every equivariant functional T. We need 

Gi = {g^G: lim inf ^(i^,^' • K) = oo}. 
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Let I denote the identity matrix in G and g\B the restriction of 5 G G to a set B G S(M"*). 
Of importance for the upper breakdown bound is 

(6) A(P) = sup{F(S) : S G = for some G Gi}, 

where in the finite sample case P will be the sample distribution -Px(„) := ^ SiLi ^"^^ 
otherwise P G 

Theorem 2.4 (Davies,Gather[H]). With the above notation and under the assumption that 
Gi ^ ^ we have 

(7) e*{T,P,d,D) < 
For the finite-sample breakdown point we have 

(8) fsbp(r„,X(„),L») < i 

If D is defined by ([3]) then Gi is the set of 5 G G such that det gg''" ^ 1. Since g in 
Example 12.11 fixes the first n — k data points in X(„) and alters the remaining k points, 
we can construct a sequence g^'^i^n) / > 1. This shows that by changing k data points 
arbitrarily we can make the new estimated value arbitrarily far from T„(x(„)) and hence 
fsbp(r„, X(„), < k/n. Note that this construction leads to a worse bound than in the 
above theorem. However, a more efficient construction giving the bound in ([8]) has a similar 
ffavor (see [7l Proof of Theorem 2.1]). 

In fSl Section 4.2] Davies and Gather discuss the following example. 

Example 2.5. (Scatter functionals and the affine group) With the sample space M"^ let the 
parameter space be given by with the natural action of the affine group g{x) = Ax + b 
inducing the action on Q. With the pseudometric in ([3]) we have 

A(P) = sup{P(i?) : B is a linear subspace of dimension < m — 1}. 

If P G ^ then A(P) = 0, but plugging in sample distributions Px(„) leads to positive values 
ofA(P.^„,). 

Note that this example includes the Gaussian graphical model for the full graph Km- In 
this paper we generalize this for any graph Q. We are looking for elements in Gi stabilizing 
some B G S(M"). The set P = {x : gx = x for some g G Gi} forms a linear subspace of 
of positive codimension. Hence, if P G ^ then for any such B G B{W^) the measure 
P(P) of B is zero. Hence A(P) = and Theorem El now states that e*(T, P, d, D) <l/2 
for any G-equivariant map T on *p. 

In the remaining of this paper we are primary interested in the finite sample case, where 
X(„) = (xi, . . . ,x„) is a random sample from the model. The following result is the third 
main theorem of the paper. 

Theorem 2.6. Let Q = ([m],P) be an undirected graph. Then 

A(P) = sup{P(P) : B is a linear subspace of dimension < Q — 1}, 



n-nA(Px(„)) + l 
2 
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where Q = maxjg[„] |{j e [m] : j ^ 
Definition \6. 5\) then 

(9) fsbp(T„,X(„,),L>) 
for any G-equivariant estimator T„. 



Moreover, if X(„^ is a generic sample (see 
^ 1 n-Q+2 



Note that if a G-equivariant map exists then by Theorem 11.21 n > Q and hence the 
right-hand side of ([9|) is always at least 1/n. If n — Q G {0, 1} then the lower and the upper 
bound on the finite sample breakdown point of T„ coincide. The upper bound on Q is m 
and it is attained only for the complete graph Km- The lower bound Q = 1 is attained for 
example for m-cycles where m > 4 and for graphs with no edges. 

Remark 2.7. If ^ is an undirected tree then Q = 2. From this it follows that for generic 
data sets fsbp(r„, X(„) , Z)) < ;^[§J- More generally whenever Q is small, like in the case 
of trees, the upper bound for the finite sample breakdown point is close to y^ln/2\. On 
the other hand if Tn is the MLE then the finite sample breakdown point is always 1/n. 
This suggests that in these cases there is a potentially big scope for improvement and more 
effort should be put into the development of more robust G-equivariant estimators. 

2 13 

Example 2.8. Let G be the graph • — • — Here Q = 2 and hence 

A(P) = ^^{^(i?) : B is a linear subspace of dimension < 1}. 

In other words this is equal to k/n where k is the maximal number of observations 
which lie on a single line through the origin. If X(„) is generic then k = 1 and by ([9]) 
fsbp(T„, X(„), L>) < ^[f J. Of course for special data sets this is not necessarily true. 
Let Q be the four-cycle then Q = 1 and hence 

A{P) = j;{P(x) : X e M-}. 

For the finite sample case this is equal to k/n where k is the maximal number of counts 
of a particular observation in X(„). If X(„) is generic then k = 1 and hence in this case 
fsbp(T„, X(„), D) < :^L^^J for any G-equivariant estimator T„. 

In the following section we analyze the symmetry group of Gaussian graphical models 
and the resulting bounds on breakdown points. 

3. The continuous part of G 

3.1. The clique poset. In this section we discuss various combinatorial constructions 
related to an undirected graph. They will all be important in our analysis of the group G. 
We first introduce some basic concepts of the theory of partially ordered sets. For further 
details see for example [21]. A partially ordered set (or poset) P = {P, <) is a set together 
with a binary relation satisfying: 

1. For all X G P, X ^ x. 

2. If X ^ y and y ^ x, then x = y. 
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3. If X ^ y and y =4 z, then x ^ z. 
If ^ satisfies 1. and 3. we say that it defines a preorder. We write x y for y ^ x and 
X -< y for X ^ y but y ^x. If ^ is a partial order then we say that y covers x (or x is 
covered hy y) \i x -< y and in addition x ^ z ^ y imphes that either z = y ov x = z. This 
concept leads to definition of the Hasse diagram of P which is a directed acyclic graph 
with the set of nodes given by P and the edges indicating the covering relation. The Hasse 
diagram is depicted so that the smallest elements are always below the larger ones. 

Let ^ be a graph with the set of vertices given by [m] and the set of edges E. We say 
that a subset C C [m] is a clique (in Q) if every two nodes in C are connected by an 
edge. We say that C is a maximal clique if C is a clique and is maximal (with respect to 
inclusion) with such a property. We denote the set of maximal cliques by C. In addition, 
for every i G [m] we define Cj to be the subset of C containing i, that is 

:= {A G C : i G ^} C 2H. 

By 1(C) the set of subsets of [m] containing C and closed under taking intersections. If 
C = {Ci, . . . , Cfc} then elements of 1(C) are of the form Hies ^« some i? C {1, . . . , k}. 
The elements of 1(C) are ordered by inclusion, which gives to 1(C) a poset structure. 

Recall the preorder defined in ([2]). The following result gives us equivalent formulation 
of the ordering relation. 

Lemma 3.1. The following are equivalent to i ^ j: 

(i) NU) U {j} C Nii) U {i} 

(ii) for all C € C if j e C then i G C, 

(iii) for all C G 1(C) if j £C then i G C, 

(iv) Cj C Cj. 

Proof. Condition (i) is just the definition of i ^ j. The implication (i) ^ (ii) follows from 
the fact that 

N{i) = {A; G [m] : 3 C G C such that i, /c G C}. 
The implication (ii) =^ (iii) is immediate. Condition (iv) is just a rewording of (ii). Finally, 
to show that (iv) implies (i) first note that necessarily j G N{i). Now let k G N{j). This 
implies that there exists C G C such that j, k and hence also i lie in C . Hence either k = i 
or k e N{i). □ 

As it was explained in the introduction, this preorder induces an equivalence relation on 
[m] hy i j if N{i) U {i} = N{j) U {j}. By i C [m] we denote the equivalence class of 
i G [m]. Now on [m]/ ^ we have an induced partial order. 

Definition 3.2. We say i =4 j for i,j G [m]/ if i =4 j for some (and hence any) i € i, 
j G j. The resulting partially ordered set (poset) on the equivalence classes is denoted by 
Pc- 

In Figure [U we give three basic graphs and Hasse diagrams of the corresponding posets 
Pc. The poset Pc was first introduced in [15j. We note in passing that not all posets Pc 
arise as Pc for some Q. A simple example is given by two posets in Figure El Now, using 
the developed theory, we analyze formally the group G defined by ([T]). 
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Figure 1. Three examples of graphs and corresponding posets Pc repre- 
sented by their Hasse diagrams. 



1 




Figure 2. Two examples of posets that cannot arise as Pc for some graph 
with the set of maximal cliques C. 



3.2. The connected component of the identity. Recall the symmetry group G of a 
Gaussian graphical model defined in the introduction. By ([T]) this is a closed subgroup 
of GLm(K) and hence a Lie group. Some basic results in the Lie group theory have been 
provided for reader's convenience in the appendix. Denote by the connected component 
of G containing the identity matrix /. By standard Lie group theory, G^ is uniquely 
determined by its Lie algebra g^. We first exploit the fact that G^ contains the set of 
all diagonal matrices T*" := (GLi(M))'" C GLm(M). Let Q = {[m],E) be an undirected 
graph and let Pc be its associated poset defined in the previous subsection. We obtain 
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the fohowing result, which also shows that the group defined here coincides with 
defined in the introduction. 

Proposition 3.3. For every i,j € [m] the element Eij lies in if and only if j ^ i. In 
particular, for every g G G^ we have gij = whenever j ^ i. 

Proof. By Lemma lA.ll it remains to check which Eij lie in or equivalently, which 
(/ + tEij) lie in G° for all t e M. We check for which i ^ j ^ [m] 

(10) {I + tEji)K{I + tE^j) G Sg, 
whenever K £ Sg. We have: 

(11) {I + tEji)K{I + tEij) = K + t{EjiK + KE,j) + t^EjiKEij. 

Denote by € M'" the unit vector with 1 on the coordinate corresponding to k £ [m]. 
Since for every k ^ I € [m] necessarily e^EjiKEijei = 0, the matrix in (jlip lies in Sg if 
and only if 

(12) el{EjiK + KEij)ei = for every (fc, /) ^ ^. 

If we take {k, I) equal to {i,j) then e]^ {EjiK + K Eij)ei = Ku and hence {i,j) needs to be 
an edge of Q. The condition in (jl2p is automatically satisfied if j ^ k,l. Now we check the 
case j = I (but i ^ k). In this case (jl2p is satisfied only if Ki^i = or equivalently (i, k) ^ E. 
In other words for every (j, k) ^ E also {k, i) ^ E, which translates into N{i)Lii C N{j)[Jj 
and consequently to j ^ i. j = k (but i ^ I) then we get the same condition. □ 

2 13 

Example 3.4. Consider a simple graph ^3 : • — • — By Proposition 13.31 the Lie algebra 
0° is generated by i^n, i?22i -E'33 together with E21 and E^i. The element £"21 lies in 
because iV(2) U 2 = {1, 2} C N{1) U 1 = {1, 2, 3}. This also shows that £^12 does not lie in 
g^. The group G^ consists of matrices of the form 

" * " 

* * , 

* * 

where the asterisk denotes an element which can be non-zero. 

Remark 3.5. The group G^ has been studied for other Gaussian models. For conditional 
independence lattice models this group was named the group of generalized block-triangular 
matrices with lattice structure (see [2^ Section 2.4]). The link between the conditional 
independence lattice models and certain Gaussian graphical models is discussed in [l] . 

4. The full structure of G 

In general G may contain elements which are not in G^ . For example for the graph ^43 
the matrix 

"100" 
1, 
1 
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represents the only non-trivial automorphism of this graph, lies also in G but it does not 
lie in G^. Recall that a graph automorphism cr of ^ = {[m], E) is a bijection of [m] such 
that {u,v) € -E if and only if {a{u), a{v)) G E. Any such automorphism gives rise to an 
element of G by taking the corresponding permutation matrix and thus Aut(^) C G. 

Recall from the introduction the definition of the induced graph G on the set of equiv- 
alence classes [m]/ ~. Note that we have a (non-canonical) embedding Aut(G, c) — > 
Aut(^) C G defined as follows. In each equivalence class of i choose an arbitrary labelling 
of the elements. Then, every automorphism of {G,c) lifts to a unique automorphism of G 
preserving the labelling. 

It is a standard result from the Lie group theory that the connected component of the 
identity G° is a normal subgroup of G. Hence, to show that G = G^ yj Aut(t7,c), in order 
to prove TheoremdH we need to show that G = G° ■ Aut(G, c) and G° n Aut(^, c) = {/}. 
For this, it suffices to construct a surjective homomorphism 

(13) a: G — > Aut{§,c), g^cTg 

such that ker((T) = G^ . To construct the homomorphism ()13p . consider now the family 

F := {a G : aJ G Sg}. 

For a ^ F and g G G we have g^aa^g G Sg so that g^a G F. Necessary and sufficient for 
a G F is that supp(a) defined by 

supp(a) := {i G [m] : / 0} 

is a clique in Q. In particular Ci (z F for all i G [m]. Hence supp(g'^ej) is a clique, which 
we denote by Rg{i). 

We use Rg also to denote the induced map Rg : 2['"1 ^ 2[™]: 

(14) Rg{A) = [jRg{i). 

Note that directly from the definition it follows that 

I I 

(15) RgiljA^) = [jRg{Ai) 

i=l i=l 

for any subsets Ai,...,Ai C [m]. Denote by Oa the |^|-dimensional linear subspace of 
M™ of all points x = (xi, . . . , Xm) with Xj = for all i G [m] \ A. Since g is invertible we 
conclude that 

(16) |-Rg(^)l > l-4| for every subset ^ C [m]. 

This follows from the fact that g^ maps the subspace Oa of M'" into O^^ (^a) so the dimen- 
sion of Oji^^A) has to be at least the dimension of Oa- 

Lemma 4.1. For every g G if G is a clique of Q, the set Rg{C) is a clique as well. 
Moreover, if C is a maximal clique then Rg{C) is maximal as well, and of the same cardi- 
nality. 
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Proof. For a generic a G F with support C the support of g^a is Rg{C). Hence this must 
be a chque. If C is a maximal clique then let D 5 Rg{C) be a maximal clique containing 
Rg{C). Since g~'^ ei = Ci for every i ^ C then, for some j G Rg{C), {i} C Rg-i[j). 
In particular C" = Rg-i{D) is a clique of Q containing C and hence necessarily C = C. 
Similarly, for every i & D, for some j € C, {i} C Rg{j) and hence Rg{C) = D. □ 

Denote by Aut(I(C),c) the subgroup of Aut(I(C)) preserving cardinality of elements of 
1(C). We have the following important result. 

Proposition 4.2. For every g & G the map M i— )• Rg{M) defines an automorphism 
Pg : 1(C) 1(C). The map p : G ^ Aut(I(C), c) given hy g ^ pg is an antihomomorphism. 
Moreover, ker p = G^. 

Proof. First we have to show that pg is a well defined order preserving automorphism of 
1(C) for every g G G. Let C = {Ci, . . . , C^} then 

M=p|Q for some A C [k]. 

ieA 

Since M is a clique, Rg{M) is a clique as well by Lemma [4.1[ Also, by (jl4p . if /c E Rg{M) = 
UjeM RgU) then there exists / G M such that /c E -Rg(0 ^-'^d i?g(/) ^ Rg[Ci) for all i G A. 
It follows that i?g(M) C (^^^^^^(Ci)- Hence 

(17) m < \Rg{M)\ < \f~]Rg{G^\, 

ieA 

where the first inequality follows from (I16p . We use now the same argument for Rg-i. 
Since C, is a maximal clique, Rg-i(Rg(Ci)) is a maximal clique. Moreover, g~'^g'^ei = Si 
for every i G [m] and hence, for some j G Rg{Ci), i G Rg~i{j), which implies that 

Rg-^Rg{Ci) = Ci 

by maximality of both cliques. It follows that 

Rg-,{[]Rg{Ci)) C [-]Rg-,Rg{Ci) = M 

ieA ieA 

and consequently 

(18) \[]Rg{C,)\ < \Rg-rif]RgiCi))\ < \M\, 

ieA ieA 

where again the first inequality follows from (fT6|) . By (fT7|) and (fTSj) we have that 

|M| = \Rg{M)\ = \f]Rg{C,)\. 

ieA 

This shows that Rg preserves cardinality of elements of 1(C). Moreover, together with the 
fact that Rg{M) C \^-^j^Rg{Ci) this implies that 

(19) Rg{[^Ci) = i^RgiC). 

ieA ieA 
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This proves that pg is a set automorphism of 1(C) preserving cardinahty. To show that 
Pg preserves the order let M d N he two elements in 1(C). Then M = Hj^^ Cj and 
N = flies Ci for B C A. By also Rg{M) C Rg{N), which proves that pg £ Aut(I(C), c) 
for every g £ G. 

To show that p : Q ^ Aut(I(C), c) defines a group antihomomorphism, it suffices to show 
that RgRh{C) = Rhg{C) for every g,h eG and C G C. If j G Rhii) for i G C and k G Rg{j) 
then k G supp{g^h^ei) = supp((%)^ei) so A; G Rhg{i) and hence RgRh{C) C Rf^giC). 
Since both sets are maximal cliques by Lemma l4.1l thev must be equal. 

We now show that ker(/9) = G^ . \i g ^ G^ then gij can be nonzero (or j G Rg{i)) only if 
j =4 i- If C is a maximal clique then j G Rg{C) only if there exists i (z G such that j ^ i. 
In particular j G C as well because C, C Cj, and hence Rg{G) = C (because Rg{G) C G 
and is maximal). This shows that li g G^ then pg is the identity, so G^ C ker(cj). 

Let now pg be the identity of 1(C). Take any i, j G [m] such that j ^ i or equivalently, 
by Lemma [STTl Cj ^ Cj. Take C G Cj \ Cj. Since Rg{C) = G then in particular j ^ -Rg(^) 
or equivalently gij = 0, which implies that g G G". This shows that also ker(p) C G^ . □ 

In the remaining part of this section we study the connection between Aut(I(C),c) and 
Aut(^,c). Since, by Lemma [3Tt j ^ i if and only if Cj C Cj then 



and hence i lies in 1(C). We say that A G 1(C) is join-irreducible if ^4 7^ and A cannot 
be expressed as S U C for C G 1(C) such that B,G C A. The set of all join-irreducible 
elements of 1(C) is denoted by J(C). The elements in 1(C), which are smaller than X i, are 
obtained by intersecting I i with some G ^ Cj. Since, for every G ^ Ci we have i ^ {I i)riG 
then ^ i G J(C). On the other hand, if j4 G J(C) then there exists i ^ A such that i ^ G 
for every C C ^ in 1(C). Suppose that there exists D £ Ci such that A <^ D. Then 
G = An D C A and i £ G which leads to a contradiction and hence if D G Cj then A<^ D 
or equivalently A C HcgC ^ ™ -'■(^)- Since i £ A then also A = PlceC ^ consequently 
every join-irreducible element of 1(C) is of the form J, i. 

Lemma 4.3. The map (p '■ i i is an order isomorphism Pq ~^ •J(C). 

Proof. Since i is join-irreducible and it does not depend on the representative of i, (j) is 
well defined. Since every A G J(C) is of the form 4- i then (p is surjective. It is also injective 
and hence it is a bijection. It preserves the ordering because i =4 j means Cj C d and 



(20) 




hence iiQij by (f20]) . 



□ 



Example 4.4. Consider the following graph: 
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6 7 




1 3 4 



Then there are six maximal chques given by Ci = {1,2,3}, C2 = {2,3,5}, C3 = {3,4,5}, 
C4 = {2, 6}, C5 = {5, 7} and Ce = {6, 7}. The poset 1(C) is given by: 

{1,2,3} {2,3,5} {3,4,5} {2,6} {5,7} {6,7} 








where the maximal element corresponds to an empty intersection and the minimal to the 
intersection of all maximal cliques. The join irreducible elements of 1(C) are depicted as 
solid nodes. Hence the poset J(C) is given by: 



{1,2,3} {3,4,5} 




• • • • • 

{2} {3} {5} {6} {7} 



Now we are ready to prove the first main theorem of this paper. 

Proof of Theorem \l.l\ We define a homomorphism G — )• Aut(G, c) as follows. Every g £ G 
defines an automorphism pg of 1(C) preserving cardinality as defined in Proposition 14.21 
This automorphism pg restricts to an automorphism of J(C) because J(C) consists of all 
the non-empty elements of 1(C) that cover at most one element in 1(C) and this property 
has to be preserved by every poset automorphism of 1(C). Therefore, by Lemma pg 
induces a bijection on [m]/ ~, which we denote by ag. More concretely ag is uniquely 
defined by 

Pgii = i o-g{i) for every i G [m]/ ~ . 
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Figure 3. The complete bipartite graph. 

Since by ([20]l we have icrg(i) = HceC ( ) ^ Pgii ^) — flceCi Pgi^) ttisn it 

follows that 

C^^Q = p,{Q := {pg{C) : C G CJ. 

Two equivalence classes i,j are linked in G if and only if there exists C G C such that 
i,j £ C or equivalently CiCiCj 7^ 0. In this case also C^g(i) l^C^g(J) ^1 which shows that ag 

induces a graph automorphism of G. The fact that this automorphism will always preserve 
the cardinality of equivalence classes follows from the fact that pg preserves cardinality of 
elements of 1(C). Denote by the homomorphism G — )• Aut(G, c) defined by 5 i-> cj"^. 
Since ker a = ker p then by Proposition 14.21 ker a = G^. □ 

5. Special graphs and small examples 

Assume that a graph Q with set of nodes given by [m] is given in form of the set C of 
its maximal cliques. The computation of G^ can be done efficiently by listing all Cj for 
i € [m]. Then by Lemma |3. II (iv) we have i ^ j if and only if Cj C d. On the other hand, 
if G is given explicitly in terms of the edge set E it is more efficient to use Lemma 13.11 (i) 
and for each i E [m] to compute N{i) U i = {j G [m] : € E} U i. The computation of 

the discrete part Aut(^, c) is in general much harder and it depends on current algorithms 
for computing automorphism groups of graphs. 

Let Sm denote the symmetric group on [m], Dm the dihedral group of graph isomor- 
phisms of an m- cycle. Also recall that ~ (GLi(M))'' denotes the group of all diagonal 
invertible k x k matrices. In Table [U we provide the full description of G for all undirected 
graphs on m = 2,3,4 vertices. 

In the rest of this section we also compute G for some interesting families of graphs. 
Consider for example the complete bipartite graph in Figure El The maximal cliques 
containing are {{ai, bj} : j = 1, . . . ,q} for every i = 1, . . . ,p and the maximal cliques 
containing bj are {{ai,bj} : i = for every j = If p = g = 1 then 

G = GL2(M). If p = 1, g > 2 then the poset Pc looks as follows. 
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Q 


Pc 


GO 


Aut(a,c) 


1 2 
• — • 


1,2 
• 


GL2(M) 


{id} 


1 2 


1 2 






• • 


• • 







A 



3 2 



12 3 
• — • — • 



12 3 



12 3 
• • • 



1,2,3 



1,2 3 
• • 

12 3 
• • • 



GL3(M) 



GL2(M) X 



'TxT ' 

3 9t-^ 2 

4 » 1 

3 2 

4 • — • 1 

3 1-1 2 

4 » 1 

3 2 

4 • • 1 

3 2 

12 3 4 
12 3 4 



12 3 4 

•— •— • • 

12 3 4 

•— • • • 

12 3 4 



1,2,3,4 



1,3 



12 3 4 
• • • • 

2,3 4 



1,2,3 4 

• • 

1 4 

T T 



1,2 3,4 



1,2 3 4 

• • • 

12 3 4 

• • • • 



* * * 



* 

* * * 

* * * 

* * 



GL. 



X 



* 

* * 
* * 
* 



X GU 



* * 
0*00 
0**0 

* 



GU 



rp4 



Table 1. Small undirected graphs Q, corresponding groups G° and 
Aut(^,c) up to isomorphism. 
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Finally, if p,q > 2 then none of the nodes are comparable. It follows that = 
Since Q = Q and Aut(^,c) = Sp x Sq, we obtain the following result. 

Proposition 5.1. Let Q he the bipartite graph in Figurel^with p, q > 2. Then 

G = TP+'^ X {Sp X Sg). 

If G is an m-cycle then if m > 4 we have that C = {{1, 2}, . . . , {m — 1, m}, {m, 1}} and, 
as above, the poset Pc is trivial with none of the nodes comparable. In this case Aut{Q, c) 
is the dihedral group Dm- If Q has no edges then Aut(^,c) = Sm- Finally, if G is the 
complete graph on m nodes then Q is a simple node with cardinality m and hence there 
are no non-trivial automorphisms of Q. We obtain the following result. 

Proposition 5.2. Let Q he an undirected graph with m vertices. Then 

• if Q is the m-cycle for m > 4. 

G = T'" X Dm. 

• if Q is a graph with m >2 vertices and no edges 

G = r X Sm' 

• if Q is the complete graph 

G = GL„(]R). 

We conclude this section with a special small graph - the bull graph - which is a graph 
on five vertices depicted in Figure |H The terminology is taken from the graph theory (see 
for example [6]). This simple graph is very special because it is the smallest graph such 
that there exists an element in Fq covering more than one element, which is relevant in 
the study of G-orbits in Sg. We say that a graph G is hull-free it does not contain a bull 
graph as an induced subgraph. We have the following result. 

Lemma 5.3. Let G he a hull-free undirected graph. Then in Pc every element covers at 
most one element. 

Proof. Suppose that k covers i and j in P^. This implies that N{k) U C N{i) U i and 
N{k) UkC N{j)Uj, where neither N{i) U i C N{j)Uj nor N{j) Uj C N{i) U i. It follows 
that we can find vertices I and m such that / G N{i) \ (iV(fc) U k) and / G N{j) \ iN{k) U k) 
such that the induced subgraph on i, j, k, I, m is the bull graph. □ 

The existence of a bull graph as an induced subgraph in G does not imply that we can 
find an element in Pc covering more than one element. A simple example is obtained by 
adding to the bull graph in Figure [4] an additional vertex and joining it with the vertex 3. 
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4 5 




3 12 



Figure 4. The bull graph and the corresponding Pc on the right. 

6. G-ORBITS IN M™ AND G-EQUIVARIANCE 

The final aim of this section is to prove Theorem II. 21 and Theorem 12.61 Before we do this 
we need to formally analyze G-stabilizers of points in M™, or more specifically, pointwise 
stabilizers of subsets B C M™. By duality between G-stabilizers and G-orbits we proceed 
with the analysis of G-orbits of M™ in Section [6Tj To complete our discussion in Section [6^2] 
we provide the conditions on the sample size for the existence of a G-equivariant estimator. 

6.1. G-orbits in the sample space. Since G contains the group of all invertible diagonal 
matrices T™, every vector x = (xi, . . . , Xm) with support A = {i : Xj 7^ 0} C [m] lies in the 
same orbit as the vector ind(x) € {0, 1}*" with the same support. Hence all that matters 
in the identification of G-orbits is the support of a vector. Recall from Section [H that for 
A C [m] by Oa we denote the set of all points in M™" with the support equal to A. 

The following lemma shows that, when searching for the supremum in ([6]), we can restrict 
to elements g S G'^ H Gi. This shows that for our analysis it is sufficient to understand 
G'^-orbits, which will be the main focus of this section. 

Lemma 6.1. // the pointwise stabiliser in G of a subset B € B{M"^) contains an element 
g such that det g 7^ ±1, then it contains an element of \ {I}. 

Proof. Let g he a pointwise stabiliser of B such that detg 7^ ±1. Then also g'' is a 
pointwise stabiliser of B. By Theorem 11.11 every g (z G can be written as g = g'a, where 
0" is a permutation matrix and g' € G^. Since Aut{Q,c) is a finite group then there exists 
/o > 1 be such that cr'" = /. In this case (7'° = g''" € G°. Because det g ^ ±1, g' ^ I, and 
hence g' e G° \ {/}. □ 

Let P be a poset and Q C P. We say that Q is a down-set if, whenever x £ Q, y & P 
and y < X, we have y & Q. Dually, Q is an up-set if, whenever x € Q, y € P and y > x, 
we have y G Q. The family of all down-sets of P is denoted by 0~''(P) and the family of 
all up-sets of P by 0^(P). They themselves are posets, under the inclusion order. For any 
A CI P define the down-set associated to 

XA := {x € P : 3a G ^ such that x < a}. 

We write J, a for i{a}, which is consistent with the notation introduced in (I20p . Dually we 
define and ta- 
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Let : 2'™'! — )• O^(Pc) be defined by A 1— >t A. This function defines an equivalence 
relation on 2H by A ~ 5 if t ^ =t ^ in 0^{Pc)- If we identify each A G 2H with the 
open subset Oa then, as the following result shows, the o-equivalence classes correspond 
to G°-orbits. 

Proposition 6.2. The -orbits ofW" are in one-to-one correspondence with the up-sets 
ofPc or equivalently with o-congruence classes in 2^"^h The orbits are 

Oa := U Ob for all A G O^(Pc). 

tB=A 

In particular dim(Oyi) = \A\ and hence the unique dense orbit is given by 

Proof, li i )^ j the element / + tEij lies in G^. Since (/ + tEij)x. differs from x only on the 
i-th coordinate where it is equal to Xi + txj we conclude that: 

(a) if i,j £ A then / + tEij stabilizes Oa] 

(b) if j £ A, i ^ A then / + tEij merges Oa and Oaui] 

(c) if j ^ A then I + tEij stabilizes Oa ■ 

Hence the orbits of correspond to subsets A C [m] closed under the above operations. 
Namely, if j £ A then i £ A for all i j- These are precisely the up-sets of P^. □ 

213 

Example 6.3. Let Q be the graph • — • — In this case we have 5 orbits of G^ represented 
by (0,0,0), (0,1,0), (0,0,1), (0,1,1), (1,0,0). This corresponds to all congruence classes 
in the Boolean lattice of subsets of {1, 2, 3}, which are marked on the corresponding Hasse 
diagram. 




The congruence class containing the set {1, 2, 3} corresponds to the unique three dimen- 
sional dense orbit containing all points with non-zero coordinates. For example, points 
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(1, 0, 0) and (1, 1, 1) are in the same G-orbit because 



1 

I—" 





" 




" 1 " 




" 1 " 


1 


1 












1 


1 





1 









1 



Finally, to understand the G-orbits it suffices to check which G'^-orbits Oa for A G 
O^(Pc) get glued by Aut(^,c). In our simple example Oi gets glued with O3. Other 
orbits are stabilized by Aut(^,c). This gives us four distinct G-orbits in M^. 

6.2. G-equivariant maps. Let Q be an undirected graph with set of maximal cliques C. 
In this section, by proving Theorem 11.21 we find the conditions which assure that there 
exists a G-equivariant estimator T„ : (M™)" Sg. This question is related to the question 
of existence of the MLE for Gaussian graphical models. If the MLE exists with probability 
1 then automatically we have a G-equivariant map T„. It turns out however that, if Q is 
not the complete graph Km, there may be cases when the MLE does not exist but still 
there are other G-equivariant maps. This will follow from Theorem 11.21 

To present known results on the existence of the MLE we first define decomposable 
graphs. 

Definition 6.4. A undirected graph Q is decomposable if it contains no induced n-cycles 
for n > 4. 

We define a decomposable cover Q* = ([m],i?*) of an undirected graph Q = {[m],E) as 
a decomposable graph satisfying E C E* . The maximal clique size of Q is q = maxcgc |C|- 
If C* is the set of maximal cliques of Q* then we also define q* = maxcgc* |G|. Let X(„) 
be a random sample of size n. Then, as it is discussed in \22\ Corollary 2.3], the MLE 
will never exists if n < q and will always exists if n > g*. Of course this latter bound is 
optimal when a decomposable cover Q* is chosen with minimal q*. This in particular gives 
the complete characterization in the decomposable case, where we can take Q* = Q. For 
non-decomposable graphs there is no known closed-form condition if q < n < q* and only 
special cases were studied (see [22] and references therein). 

As indicated in Theorem 11.21 the characterisation of the existence of G-equivariant esti- 
mators has an elegant closed form. We now prove this result. 

Definition 6.5. We say that some property holds for generic /c-tuples (xi, . . . , x^) G (M™)*^ 
if it holds for (xi, . . . outside the zero set of some non-zero polynomial. 

Note that if a property holds for generic /c-tuples, then it holds with probability one for 
the random sample X(„) drawn from any non-degenerate continuous probability distribu- 
tion. 

Theorem 6.6. The minimal number k for which the pointwise stabiliser in of generic 
k -tuple xi, . . . , Xfc G M™ consists entirely of determinant- one matrices equals maxjgf^^j] \ii\- 
For that value of k the stabiliser of k generic vectors is, in fact, the trivial group {/}. 
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Proof. The condition that g €z G fixes a single vector x € translates into m Unear 
conditions on the entries of g, namely: 



The i-th condition concerns only the entries in the i-th row of g. We therefore concentrate 
on that single row of g, and consider the entries gij, i )^ j as variables to be solved from 
the linear equations above as x ranges through the given /c-tuple of vectors. Since the 
given A:-tuple is generic, those equations are linearly independent as long as k is at most 
the cardinality of J,*- Hence they determine the i-th row uniquely as soon as k is at least 
that number. Hence as soon as k is at least the maximal cardinality of the sets over all 
i the pointwise stabiliser of the k vectors will be trivial. 

What remains to be checked is that for smaller k the pointwise stabiliser of generic 
xi,...,Xfc does not consist entirely of determinant-1 matrices. This is easiest seen by 
considering the Lie algebra of that pointwise stabiliser, which is the set of matrices A in 
the Lie algebra of G satisfying the linear conditions Axi = . . . = Ax/^ = 0. Let i be a 
row index for which has more than k elements. Then the linear conditions on A do not 
fix the i-th row of A uniquely. Moreover, by genericity, they do not even fix the diagonal 
entry An uniquely. As a consequence, they do not determine the trace of A uniquely. This 
shows that the Lie algebra of the pointwise stabiliser is not contained in the Lie algebra of 
trace-zero matrices (see Appendix |A]). But then the pointwise stabiliser is not contained in 
the Lie group of determinant-one matrices. □ 

Now we are ready to prove Theorem ll.2l which is the second main theorem of the paper. 

Proof of Theorem M.Si The G-equivariance of T implies that T{g ■ X(„)) = g ■ T(x(„)) for 
every g £ G and X(„-) S (M™)". In particular, the G'^-equivariance implies that for every 
X(„) G (M™-)"' the G*^-stabilizer of X(„) is contained in the G^-stabilizer of r(x(„)): 

(21) G° , < V 

Note that since r(x(„)) G Sg, its stabilizer is a generalized orthogonal group and hence it is 
a compact group. On the other hand, by (the proof of) Theorem 16.61 for a generic n-tuple 
X(„) is not compact unless it is trivial, which happens only when n > maxjg[„] Hence, 
if n < maxjg[„] \ this necessarily contradicts (|2ip and then there are no G^-equivariant 
maps and in particular there are also no G-equivariant maps. 

Now we show that n > maxjg[„] \-lj\ is also a sufficient condition for the existence of 
an equivariant map T. It sufficient to show this for n = max^gj^] In this case the 

G^-stabilizer of a generic point X(„) is trivial and hence a generic G^-orbit in (M™)" has 
dimension dimG = dimG*^ = ■ i ^ The dimension of the ambient space is 

equal to mn. Let 

di := n — \ > 

for every i G [m]. Then di is equal to the codimension of a generic G°-orbit in (M™)''. 




Xi for i = 1, . . . , m. 
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For each i E [m] fix a generic linear subspace Lj of of dimension dj. Do it in such a way 
that Li is the same for ah i in a single orbit under the group of graph automorphisms. Let 
L = HiGH ^* — (^")'" then L can be identified with a subspace of (M'")" by transposition. 
For generic X(„) the condition that the n-tuple (((7x1)4, . . . , {gxn)i) lies in Lj imposes 
linear constraints on the i-th row of g. Hence there is a unique solution for that i-th row. 
It follows that L intersects a generic G'^-orbit in a unique point. 

Since L is stable under the group automorphism group Aut(^) then we can consider the 
quotient L/Aut{Q). The induced projection from (M™-)" onto L/Aut{Q) is denoted by tt. 
By construction vr is G-invariant. Take now any map T„ : L/Aut(ty) — )• Sg . We extend the 
map for every X(„) € (M™)" by taking T„(x(„)) := g ■ r„(7r(x(„))) where X(„) = 57r(x(„)). 
This map is defined almost everywhere and is G-equivariant by construction. Finally, the 
equation maxjgj^] \ = max^gj^^j) |j4| follows directly from Lemma 14.31 □ 

By Lemma 14.31 Q = maxjgj^j j is equal to the cardinality of the biggest element of 
J(C). For this reason we call Q the irreducible maximal clique size. If q is the maximal 
clique size then of course q > Q. 

Proof of Theorem \2.6i Note that if a subgroup H C is a pointwise stabilizer of B C 
then it is also a pointwise stabilizer of the smallest linear subspace containing B. Hence 
the subsets B in the definition of A(P) in ([6]) can be taken to be in the family of all linear 
subspaces of M"*. 

By Theorem 16.61 the maximal number k such that the set of k generic vectors has a 
pointwise stabilizer not consisting entirely of determinant-one matrices (hence G^CiGi 7^ 0) 
equals Q — 1. This implies that the maximal dimension of a linear subspace with a pointwise 
stabilizer not consisting entirely of determinant-one matrices is Q — 1 as well. This gives 
the result by the definition of A(P) in The last statement of the theorem follows from 
Theorem 12.41 and the fact that if X(„) is generic then A(Px(„)) = (Q — □ 

7. G-ORBITS IN THE CONE Sg 

Given an undirected graph Q = {[m],E) we have determined the group G C GL^^ (^) 
of all invertible linear maps M™" stabilizing the cone Sg , or, equivalently, stabi- 

lizing the vector space Sg of symmetric matrices with zeroes on the off-diagonal entries 
corresponding to non-edges of Q. To complete our geometric description of the group G 
we need to understand the structure of G-orbits on Sg . The following result is important 
for statisticians as it tells us when the graphical Gaussian model forms a transformation 
family. 

Theorem 7.1 (Theorem 2.2, [E]). Let Q = [[m],E) he an undirected graph. There is 
exactly one orbit of the G-action on Sg if and only if for any two neighbors i,j G [m] 

either i =4 j or j ^ i; or equivalently if Q does not contain • — • — • — • as an induced 
subgraph. 

From the inferential point of view it is important to analyze also other cases, when the 
action of G on St is not transitive and hence the model forms a composite transformation 
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family. In this section we answer the first natural question about the generic dimension of 
G-orbits on Sg and Sg (this generic dimension, attained by symmetric matrices in some 
open dense subset, coincides on these two sets). Standard Lie group theory implies that this 
dimension equals dim G minus the dimension of the stabilizer of a generic matrix K £ Sg. 
Thus we set out to determine the dimension of that stabilizer. 

We do this in the following slightly larger generality. Let Q denote the graph induced by 
Q on the set C of equivalence classes of the pre-order ^ and let c : C — >■ N, i q be the 
coloring mapping an equivalence class to its cardinality. We write i ^ j to express that i 
and J arc connected in Q. Let < be a partial order on C which is compatible with the edge 
set of Q in the sense that i < j =r- N(j) U {j} C N{i) U {i}. Note that we do not require 
the implication here, so < may have less comparable elements than the partial order on 
C coming from the pre-order ^. To every i e C we associate a vector space Ui := and 
we set U := 0jg(7 ^i- want to determine the (dimension of the) stabilizer of K in the 
group 

H:= {geGL{U)\gUiC^Uj}, 

j>i 

which, if < comes from :<, is nothing but the identity component G° of G. The final answer 
is as follows. 

Theorem 7.2. The dimension of the stabilizer Hk in H of a generic matrix in Sg (or 
Sg ) equals Y^^^c C2*) ' where Ui is defined by 

Hi := max < 0, Cj — j cj 

In words: starting from Q, one deletes all edges between vertices that are comparable in 
the partial order < , and one subtracts from Cj the sum of the cj for all neighboring j in the 
new graph. If the result is non-negative, then this is n^; otherwise, is zero. The expression 
above suggests that the identity component of Hk is a product of special orthogonal groups 
of spaces of dimensions rtj, which is indeed what the proof will show. In the following proof 
we will identify K with the bilinear form on U defined by K{'x,y) := 'x^Ky, and terms 
such as orthogonal complement and orthogonal group will refer to that form or restrictions 
of it. 

Proof. We first prove the statement in the case where < is trivial, so that H consists of all 
linear maps h G GL(f/) that are block diagonal in the sense that they preserve the direct 
sum decomposition U = 0L'j. For each i G C let U- C Ui denote the intersection with 
Ui of the orthogonal complement of ©^yj Uj. Since the Uj with j i are perpendicular 
to Ui, the space also equals the space of vectors in Ui perpendicular to all spaces Uj 
with j ~ i. By genericity of K, these orthogonality conditions defining U- are as linearly 
independent as possible, and it follows that the dimension of U- is n^. Given any linear 
map h in 0(U-), i.e., preserving the restriction of K to U^, we extend it by the identity 
on the orthogonal complement of U^. The resulting map, still denoted h, stabilizes K by 
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construction, and we claim that it also maps each Uj into itself. For j ^ i this is clear since 
Uj is perpendicular to and hence h is the identity on Uj; for j = i this is clear since h 
preserves C// and acts as the identity on the orthogonal complement of U- in C/j. Thus h lies 
in Hk- Doing this for all i yields a product of orthogonal groups, one for each Ul, which 
is contained in Hk- The dimension of this product is the expression in the theorem. We 
claim that the identity component of Hk equals the identity component of this product 
(which is the product of special orthogonal groups). 

To see this, we first discuss the case where Q has two vertices connected by an edge. 
Label the vertices 1 and 2, with cardinalities ci, C2, respectively. Without loss of generality 
we assume that ci > C2. Then any g € Hk must stabilize U[ defined as above (note that 
U2 is zero), and hence also the orthogonal complement U" of U[ in Ui. Note that 
and U2 have the same dimension. The restriction of g to U" ® U2 is block diagonal with 
blocks gi G 0{Ui) and g2 G 0(6^2) and moreover satisfies K{gixi, g2X.2) = i^(xi,X2) for 
xi € U'l and X2 € U2- We claim that for generic K the group of such pairs (51,52) is 
zero-dimensional. It suffices to prove this for a specific and we take K, or rather its 
restriction to U'{ (BU2, to have the block structure 

'I I' 
I D 



K 



where / is an identity matrix of size dim = dim U2 = C2 and Z? is a diagonal matrix all 
of whose entries are distinct. Now the Lie algebra of the group of pairs (51,52) as above 



A2) of C2 X C2-matrices satisfying 





"0 


0" 











consists of all block diagonal matrices A = diag(Ai, 
AJ'K + KA = 0. Substituting the blocks yields 

'A{ + Ai Aj + A2 
Al + Ai A^D + DA2 

This means that Ai is skew-symmetric (in the ordinary sense, i.e., its (i,j)-cntry is minus 
its (j, i)-entry), and hence that A2 = —Aj = Ai. But A2 also satisfies {A2)jiDjj = 
—Dii{A2)ij. Using that the Da are distinct one finds that A2 = Ai = 0. Hence said Lie 
algebra is zero, and the identity component of the group of pairs (51,52) is trivial, hence 
the identity component of Hk reduces to SO(f/(), as claimed. 

Next we discuss case where Q is general but < is still trivial. As mentioned earlier, this 
means that H consists of block diagonal linear maps; denote by hi the restriction oi h E H 
to Ui. We want to prove that for any h in the identity component of Hk, each hi lies in 
S0(?7j') in the sense that that it is the identity on the orthogonal complement of Ul inside 
Ui- For each j ~ z the pair {hi, hj) stabilizes the restriction K' of K to Ui® Uj, and lies in 
H'j^,, where H' Q H is the subgroup GL(C/j) x GL{Uj). By the discussion of the two-vertex 
case above, we find that hi is the identity on the orthogonal complement in Ui of U R Uj". 
Note that, by genericity of K, this orthogonal complement is all of Ui if Cj < Cj, and of 



dimension q — (cj 



Cj otherwise. Varying j over the neighbours of i, we find that hi 



is the identity on the sum over j of all these orthogonal complements, which is the same 
thing as the orthogonal complement in Ui of f]j^i{Ui fl Uj-) = U[. This proves the theorem 
in the case where < is trivial. 
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It remains to prove the theorem in the case where < is non-trivial. Then any h £ H 
stabihzes the spaces U>i := 0j>j t^j and C/>j := 0j>jf^j. If h also stabilizes K, then 
it stabilizes the orthogonal complement Vi := U^^ fl U>i of C/>j inside Note that Vi 
has dimension a, and we argue that we can replace the Ui by the Vi. Indeed, for any 
pair i,j £ C consider the restriction of the the bilinear form K io Vi x Vj. If i 9^ j, then 
we claim that this restriction is zero. Indeed, each element of 1/^ is a linear combination 
of elements of the spaces Ui' with i' > i, and similarly for the elements of Vj. Now if 
some i' > i were connected to some / > j, then by the compatibility of the partial order 
with the edge set we find that also i ~ / (since i < i') and hence i ~ j (since j < j'), a 
contradiction. Hence no i' > i is connected to any j' > j, and therefore the restriction of 
K to U>i X U>j is zero. In particular, K restricts to zero on Vi x Vj. Next, if i ~ j but i 
and j are comparable in the partial order, then by construction Vi is perpendicular to Vj. 
Hence only if i ~ j and i,j are not comparable can K restrict to something non-zero (and 
generic) on Vi x Vj. Thus upon replacing the Ui by the Vi and deleting from Q the edges 
among vertices comparable in the partial order we are reduced to the previous case, where 
< is trivial. This concludes the proof of the dimension formula. □ 

Note in particular that Theorem 17.21 confirms that, in the situation of Theorem 17. 1^ the 
dimension of the stabilizer of a generic matrix in Sg is Yliec (2') • Consequently, the di- 
mension of a generic orbit equals dim G minus this expression, which under the assumption 
of Theorem l7.1l is readily seen to equal the dimension of the model, in accordance with the 
fact that Sg is a single orbit. Another example is given by the bull graph. 

Example 7.3. Consider the bull graph in Figure H] but with each vertex now representing 
an equivalence class in C with cardinality for i = 1, . . . , C5. In this case the only pair 
of connected but not comparable vertices is (1,2). With the convention that C^) = if 
m < 0, Theorem 17.21 gives that the dimension of the stabilizer of a generic matrix in Sg is 

Example 7.4. Let now ^ be a tree, where each vertex v represents an equivalence class 
with cardinality c^. In this case the dimension of the stabilizer of a generic matrix in Sg 
is 

E ((°-r')+C"r'))+ E (2)- 

(?i,-u)ginncr deleaves 

where the first sum is over all the inner vertices of G and the second sum is over all the 
leaves (vertices of valency 1) of ^. In particular if for some c we have that Cj = c for all 
i € C then this formula degenerates to /(g), where I is the number of leaves. 

8. Discussion 

Given an undirected graph G = {[m], E) we have determined the group G C GLm(K) of 
all invertible linear maps M™" — )■ stabilizing the vector space Sg of symmetric matrices 
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with zeroes on the ofF-diagonal entries corresponding to non-edges of Q. This helped us 
to analyze bounds on the finite sample breakdown points for G-equivariant estimators 
for Gaussian graphical models and generic data sets. Note that in practice often X(„) 
is not generic. Consider for example the problem of modeling the height of people with 
measurements taken in centimeters. In a typical sample for this case we can expect many 
repeated data points. In this case a linear subspace of dimension Q — 1 may contain more 
than Q — 1 data points which will lead to higher values of A(Px„)- It would be helpful 
to have efficient ways of computations for non-generic data sets. Note also that the proof 
of Theorem 11.21 gives a general construction of a G-equivariant estimator. It would be 
interesting to see how this construction helps to construct robust estimators with some 
explicit high lower bounds on the finite sample breakdown points. 

Appendix A. Matrix groups and Lie algebras 

In this appendix we collect some basic facts about Lie groups that are used in the paper. 
For details see for example [3j . We only need the notion of a matrix Lie group, which is a 
closed subgroup G of the group GLm(K) of invertible real m x m-matrices. It follows that 
G is, in fact, a smooth submanifold of GLm(M). Two examples featuring in the paper are 
the group SLm(IR) consisting of determinant-one matrices and the generalized orthogonal 
groups 0/^(M), one for every positive definite symmetric matrix K, consisting of matrices 
g such that g^Kg = K (we use notation 0(M) if K is generic - see Section [7]). Among 
these, SLm(K) is connected, while Oi^(M) has two connected components, distinguished by 
the value ±1 of the determinant. The groups Oi^ (M) are compact, because they are closed 
and bounded in the Euclidean norm on the space of m x m-matrices (for this one needs 
the fact that K is positive definite). Two further classes of examples are T™, the group 
of diagonal invertible m x m-matrices, and the symmetry groups of Gaussian graphical 
models discussed in the paper. 

To understand a matrix Lie group G it helps to understand its tangent space at the 
identity matrix. This vector space of m x m-matrices is usually denoted g, and has the 
structure of a matrix Lie algebra, i.e., it is closed under taking matrix commutators. A basic 
fact used in the paper is that 3 uniquely determines the connected component G^ of G that 
contains the identity matrix. Using the fact that the derivative of the determinant function 
at the identity matrix is the trace, one finds that the Lie algebra of SLm,(IR) consists of all 
trace-zero matrices. Similarly, by taking derivatives of the equations defining Oi^(M) one 
finds that its Lie algebra consists of all matrices A satisfying A^K + KA = 0. 

The following well-known lemma connects certain matrix Lie groups to the combinatorics 
of preorders, and plays a central role in the paper. For completeness, we give a proof. 

Lemma A.l. Let H C GLm(M) be a matrix Lie group containing the group T™. Then the 
Lie algebra of H has a basis consisting of matrices Eij with {i,j) running through some 
subset I of [m] x [m]. Moreover, the set L defines a preorder on [m] in the sense that {i,i) 
lies in L for all i € [m] and {i,j),{j,k) & L ^ {i,k) & L. Conversely, the Eij with {i,j) 
running through any set I Q [m] x [m] defining a pre-order span the Lie algebra of a closed 
subgroup of GLm(K) • 
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Proof. Let t ~ M"* denote the Lie algebra of T"*, which is spanned by the En for i e [m]. 
Then t is a Lie subalgebra of [}. Let A = X^j^j o-ijEij be any element of f) with zero entries 
on the diagonal. We show that E^i G f) for every (fc, /) such that aki ^ 0. We have 

[Ell, [Ekk,-^]] = —o-lkEik — dkiEki G f). 

If only one of Uki and aik is non-zero, then the result follows immediately. If both are 
non-zero, then note that 

[Ekk, [Ell, [Ekk, A]]] = aikEik - akiEki- 

Since f) is a linear subspace, we conclude that both Eki and Eik lie in [}. 

Thus [) is spanned by matrices Eij with (i, j) running through some subset / of [m] x [m]. 
The inclusion t C f) implies that G / for all i. Now if Eij,Ejk G t) with k ^ i, then 
[Eij, Ejk\ = Eik so that (i, k) G /, as well. This yields the preorder property of /. 

Finally, if / C [m] x [m] determines a pre-order, then the set of all invertible matrices 
g with Qij = unless (i, j) G / form a matrix Lie group (by a computation similar to the 
one just performed), whose Lie algebra is spanned by the E^j with (z, j) G /. □ 

In the paper we also implicitly make use of the fact that the matrix exponential map 
maps the Lie algebra of a matrix Lie group into the group, but only in the basic case where 
exp(tEy) = l + tEij. 
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